Image registration method and model training method thereof

ABSTRACT

Disclosed in the present disclosure are an image registration method and a model training method thereof. The image registration method comprises obtaining a reference image and a floating image to be registered, performing image preprocessing on the reference image and the floating image, performing non-rigid registration on the preprocessed reference image and floating image to obtain a registration result image, and outputting the registration result image. The image preprocessing comprises performing, on the reference image and the floating image, coarse-to-fine rigid registration based on iterative closest point registration and mutual information registration. The non-rigid registration uses a combination of a correlation coefficient and a mean squared error between the reference image and the registration result image as a loss function. Further disclosed in the present disclosure are an apparatus and a system for image registration and a computer-readable medium corresponding to the method. The present disclosure can realize precise and efficient image registration with high applicability between images of different time, different modalities, or different sequences.

TECHNICAL FIELD

The present disclosure relates to the technical field of imageprocessing, and more specifically, to an image registration method and amodel training method thereof.

BACKGROUND

Image registration is the basis of researches on, for example, imagefusion, image reconstruction, matching of an image with a standard map,and quantitative image analysis. Image registration enablescorresponding points of two images to be consistent in spatial positionand anatomical position under a given similarity measurement by means ofan optimal spatial transformation. The image registration plan cansufficiently utilize various information contained in images ofdifferent time, different modalities, or different sequences, andprovide a basis for subsequent advanced image analysis (for example,medical efficacy evaluation, image-guided therapy, multi-sequence imagesegmentation, and merge of different modalities). Image registration maybe divided into rigid registration and non-rigid registration dependingon the transformation mode. Rigid registration means that the distancebetween any two points in an image remains unchanged before and aftertransformation, except that translation and rotation of coordinate axesoccur. Rigid registration applies only to registration withoutdeformation or rigid bodies. Rigid registration cannot meet clinicalneeds in many cases. Since many deformations are non-rigid andnon-linear in nature, many important clinical applications neednon-rigid transformation to describe a spatial relationship betweenimages. At present, conventional rigid registration methods andnon-rigid registration methods have some problems to be solved, forexample, poor applicability: a method or a set of parameters appliesonly to a specific modality or even a specific data set; slow processingspeed: since optimal parameters are searched for using iterativeoptimization in conventional registration methods, slow processing speedis caused, and it is difficult to apply the methods to real-timescenarios; or poor precision due to local extrema.

For example, lung cancer is one of the most common malignancies in theworld, having the highest morbidity and mortality among malignancies andrecognized as a killer of human health. Immunotherapy is a treatmentthat remodels the immune system of a tumor patient to kill tumor cells.Due to its small toxic and side effects and long-lasting efficacy,immunotherapy has been used in lung cancer treatment. In addition,objectively and precisely evaluating the efficacy of tumor treatment isof important clinical significance in which imaging examination such asCT and MR plays an important role. The efficacy of immunotherapy wasevaluated for imaging examination, and the RECIST Working Group and theImmunotherapy Subcommittee thereof published the iRecist standard in2017. In order to use iRecsit or other response evaluation criteria toprecisely and quantitatively evaluate the efficacy, an effective imageregistration method is an essential link. However, affected by thecomplex nature of image data after immunotherapy and the irregulardeformation caused by the spontaneous movement of lung organs, existingimage registration methods need to be improved in precision andefficiency of completing image registration.

Accordingly, there is a need in the art for an improved imageregistration method.

SUMMARY

In one aspect of the present disclosure, an image registration method isprovided. The method may comprise: obtaining a reference image and afloating image to be registered; performing image preprocessing on thereference image and the floating image, wherein the image preprocessingmay comprise rigid registration based on iterative closest pointregistration and mutual information registration; performing non-rigidregistration on the preprocessed reference image and the preprocessedfloating image to obtain a registration result image; and outputting theregistration result image.

In another aspect of the present disclosure, a rigid registration methodfor images based on iterative closest point registration and mutualinformation registration is provided. The method may comprise:performing coarse registration about contour point data sets on areference image and a floating image to be registered by using iterativeclosest point registration, so as to obtain first transformationparameters; optimizing registration between the reference image and thefloating image by using mutual information registration based on thefirst transformation parameters, so as to obtain second transformationparameters; and registering the reference image and the floating imagebased on the second transformation parameters.

In yet another aspect of the present disclosure, a method for training anon-rigid registration model for images is provided. The method maycomprise: inputting a preprocessed reference image and a preprocessedfloating image to a U-Net to obtain spatial transformation parameters;inputting the spatial transformation parameters to a spatialtransformation network, and performing spatial transformation and aninterpolation operation on the preprocessed floating image, so as toobtain a registration result image; calculating a loss function valuebetween the reference image and the registration result image by using aloss function, wherein the loss function may comprise both a correlationcoefficient and a mean squared error between the reference image and theregistration result image; and repeating the aforementioned steps apredetermined number of times for iterative training or until thenon-rigid registration model converges.

In yet another aspect of the present disclosure, a computer-readablemedium is provided, having instructions thereon, wherein when executedby a processor, the instructions cause the processor to perform thesteps of any of the methods described above.

In yet another aspect of the present disclosure, an image registrationapparatus is provided, wherein the image registration apparatus maycomprise a device for implementing the steps of any of the methodsdescribed above.

In yet another aspect of the present disclosure, a system for imageregistration is provided. The system may comprise: a medical imagingapparatus, the medical imaging apparatus being configured to perform animaging scan to generate a medical image; a storage apparatus, thestorage apparatus being configured to store the medical image; and amedical imaging workstation or a medical image cloud platform analysissystem, wherein the medical imaging workstation or the medical imagecloud platform analysis system may be communicatively connected to thestorage apparatus and comprise a processor, the processor may beconfigured to perform the steps of any of the methods described above.

These and other features and aspects of the present disclosure willbecome clearer through the detailed description with reference to thedrawings below.

BRIEF DESCRIPTION OF THE DRAWINGS

To obtain a better understanding of the present disclosure in detail,please refer to the embodiments for a more detailed description of thepresent disclosure as briefly summarized above. Some embodiments areillustrated in the drawings. In order to facilitate a betterunderstanding, the same symbols have been used as much as possible inthe figures to mark the same elements that are common in the variousfigures. It should be noted, however, that the drawings only illustratethe typical embodiments of the present disclosure and should thereforenot be construed as limiting the scope of the present disclosure as thepresent disclosure may allow other equivalent embodiments. In thefigures:

FIG. 1 schematically shows an exemplary flowchart of an imageregistration method according to an embodiment of the presentdisclosure.

FIG. 2 schematically shows an exemplary flowchart of a rigidregistration method for images based on iterative closest pointregistration and mutual information registration according to anembodiment of the present disclosure.

FIG. 3 schematically shows an exemplary flowchart of a method fortraining a non-rigid registration model for images according to anembodiment of the present disclosure.

FIGS. 4A and 4B respectively schematically show exemplary fusion effectsbefore and after image registration is performed according to anembodiment of the present disclosure.

FIGS. 5A and 5B respectively schematically show exemplary fusion effectsbefore and after image registration is performed according to anotherembodiment of the present disclosure.

FIG. 6 schematically shows an example of an electronic apparatus forperforming an image registration method according to an embodiment ofthe present disclosure.

FIG. 7 schematically shows an exemplary block diagram of an imageregistration apparatus according to an embodiment of the presentdisclosure.

FIG. 8 schematically shows an exemplary block diagram of a system forimage registration according to an embodiment of the present disclosure.

It can be expected that the elements in one embodiment of the presentdisclosure may be advantageously applied to the other embodimentswithout further elaboration.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Specific embodiments of the present disclosure will be described below.It should be noted that in the specific description of the embodiments,in order to enable a concise description, it is impossible to provideexhaustive detailed description on all features of the actualembodiments in this specification. It should be understood that in theactual implementation of any of the embodiments, as in the process ofany engineering project or design project, a variety of specificdecisions are often made in order to achieve the developer's specificobjectives and meet system-related or business-related restrictions,which will vary from one embodiment to another. Moreover, it can also beunderstood that although the efforts made in such development processmay be complex and lengthy, for those of ordinary skill in the artrelated to content disclosed in the present disclosure, some changes indesign, manufacturing, production or the like based on the technicalcontent disclosed in the present disclosure are only conventionaltechnical means. The content of the present disclosure should not beconstrued as insufficient.

Unless otherwise defined, the technical or scientific terms used in theclaims and the specification are as they are usually understood by thoseof ordinary skill in the art to which the present disclosure pertains.The words “first,” “second” and similar words used in the specificationand claims of the patent application of the present disclosure do notdenote any order, quantity or importance, but are merely intended todistinguish between different constituents. “One,” “a(n)” and similarwords are not meant to be limiting, but rather denote the presence of atleast one. The word “include,” “comprise” or a similar word is intendedto mean that an element or article that appears before “include” or“comprise” encompasses an element or article and equivalent elementsthat are listed after “include” or “comprise,” and does not excludeother elements or articles. The word “connect,” “connected” or a similarword is not limited to a physical or mechanical connection, and is notlimited to a direct or indirect connection.

The images described herein may be images of various objects including,but not limited to, images of anatomical structures (such as lungs andchests) of human patients and animals, articles (such as parts), orvarious foreign objects (such as dental implants, stents, or contrastagents) existing in the body. Further, the images described herein maybe images of various modalities including, but not limited to, imagesgenerated by a computed tomography (CT) apparatus, a magnetic resonanceimaging (Mill) apparatus, a C-arm imaging apparatus, a positron emissiontomography (PET) apparatus, a single photon emission computed tomography(SPECT) apparatus, or any other suitable imaging apparatus. Theembodiments of the present disclosure can realize precise and efficientimage registration with high applicability between images of differenttime, different modalities, or different sequences.

In the embodiments of the present disclosure, the “contour point dataset” of an image refers to a data set of all pixels on an outer contourof a target object (such as a lung organ) in the image. The term“iterative closest point registration” used herein refers to imageregistration based on an iterative closest point algorithm, and “mutualinformation registration” refers to image registration based on a mutualinformation algorithm.

Now referring to FIG. 1, FIG. 1 shows a flowchart of an exemplary imageregistration method 100 according to an embodiment of the presentdisclosure. The image registration method 100 starts at step 120. Instep 120, a reference image and a floating image to be registered may beobtained. In various embodiments of the present disclosure, thereference image may be an image selected from various obtained images(such as images of different periods, different modalities, or differentsequences) of an object to serve as a reference for image registration.For example, in registering CT images of the same patient scanned atdifferent periods, the image of the first scan is usually selected asthe reference image.

Then, in step 140, the image registration method 100 may includeperforming image preprocessing on the obtained reference image andfloating image. In an embodiment, the preprocessing of the referenceimage and the floating image may be performed at the same time. Theimage preprocessing described herein is a consistent preprocessingprocess for the reference image and the floating image in terms ofoperation steps, operation parameters, and so on, for example, using thesame pixel pitch and the same normalization equation. In this way, theproblem of poor registration caused by the complex nature of image data(such as medical images obtained from medical imaging scans of patientsor other objects subjected to immunotherapy) or different qualities ofimages acquired by different apparatuses can be alleviated oreliminated, thereby improving the applicability of the imageregistration method described herein.

The image preprocessing 140 may include performing rigid registration146 on the reference image and the floating image. The rigidregistration may typically include iterative closest point (ICP)-basedrigid registration and mutual information (MI)-based rigid registration.The conventional iterative closest point-based rigid registrationperforms matching only on contour point data sets of the reference imageand the floating image, and a small amount of data is calculated. Thus,the registration has high speed but relatively low precision. Moreover,the conventional mutual information-based rigid registration performs aplurality of iterative searches for optimal transformation parameters,and a joint distribution of the reference image and the floating imageneeds to be repeatedly calculated in the process of search andcalculation. Thus, the registration has high precision, but has problemsof being time-consuming and easily falling into local extrema. In therigid registration method based on iterative closest point and mutualinformation described herein, transformation parameters output initerative closest point registration are used as initial values ofmutual information registration to combine the two to achievecoarse-to-fine rigid registration of the images. The transformationparameters obtained through iterative closest point registration areused as the initial values of mutual information search, so that thesearch range of globally optimal transformation parameters can besignificantly narrowed to avoid falling into local extrema, therebyachieving the purpose of improving registration precision; besides, thenumber of searches can be reduced in this manner to achieve the purposeof increasing registration speed. An exemplary rigid registration method200 for images incorporating iterative closest point-based registrationand mutual information-based registration according to an embodiment ofthe present disclosure is described in more detail below with referenceto FIG. 2.

The rigid registration method 200 for images based on iterative closestpoint and mutual information starts at step 230. In this step, the rigidregistration method 200 may include performing coarse registration aboutcontour point data sets on a reference image and a floating image to beregistered by using iterative closest point registration, so as toobtain first transformation parameters, as shown in block 230 of FIG. 2.

In some embodiments, the respective contour point data sets of thereference image and the floating image may be extracted through amarching cubes (MC) algorithm. In some embodiments, the firsttransformation parameters may be three-dimensional transformationparameters. In a three-dimensional embodiment, the first transformationparameters may include a pixel translation amount in an x-axisdirection, a pixel translation amount in a y-axis direction, a pixeltranslation amount in a z-axis direction, a central rotation angle in anaxial plane (xy plane), a central rotation angle in a sagittal plane (xzplane), and a central rotation angle in a coronal plane (yz plane). Insome other embodiments, the first transformation parameters may betwo-dimensional transformation parameters. In a two-dimensionalembodiment, the first transformation parameters may include a pixeltranslation amount in an x-axis direction, a pixel translation amount ina y-axis direction, and a central rotation angle in an xy plane.

In the embodiment of the present disclosure, after the iterative closestpoint registration, as shown in block 250 of FIG. 2, the rigidregistration method 200 may include optimizing registration between thereference image and the floating image by using mutual informationregistration based on the first transformation parameters, so as toobtain second transformation parameters.

Specifically, in the embodiment, spatial transformation may be performedon the floating image first by using the obtained first transformationparameters as initial values of mutual information registration. Thespatial transformation performed on the floating image can align overallcontours of the floating image and the reference image. In an example,the spatial transformation may include a translation operation and/or arotation operation on the floating image. The translation operation mayinclude translation operations of the floating image in the x-, y-and/or z-axis direction. The rotation operation may include rotationoperations of the floating image in the axial plane (xy plane), sagittalplane (xz plane), and/or coronal plane (yz plane).

After the spatial transformation, an interpolation operation may beperformed on the floating image. In an embodiment, the interpolationoperation performed on the floating image may be, for example, nearestneighbor interpolation, partial volume (PV) interpolation, linearinterpolation, B-spline interpolation, bilinear interpolation, and/orbicubic interpolation.

The mutual information registration may then further include calculatinga mutual information value between an obtained interpolation result andthe reference image, so as to obtain the second transformationparameters corresponding to the mutual information value. Like the firsttransformation parameters, in some embodiments, the secondtransformation parameters may also be three-dimensional transformationparameters. In a three-dimensional embodiment, the second transformationparameters may include a pixel translation amount in an x-axisdirection, a pixel translation amount in a y-axis direction, a pixeltranslation amount in a z-axis direction, a central rotation angle in anaxial plane (xy plane), a central rotation angle in a sagittal plane (xzplane), and a central rotation angle in a coronal plane (yz plane). Insome other embodiments, the second transformation parameters may betwo-dimensional transformation parameters. In a two-dimensionalembodiment, the second transformation parameters may include a pixeltranslation amount in an x-axis direction, a pixel translation amount ina y-axis direction, and a central rotation angle in an xy plane.

After the mutual information registration is performed, the rigidregistration method 200 may include registering the reference image andthe floating image based on the obtained second transformationparameters, as shown in block 270 of FIG. 2.

In a preferred embodiment of the present disclosure, the rigidregistration method 200 based on iterative closest point registrationand mutual information registration may include, after the mutualinformation registration is performed, determining whether thecalculated mutual information value converges or whether the iterativeclosest point registration and the mutual information registration arerepeatedly performed a predetermined number of times. The “mutualinformation value converges” mentioned herein refers to that adifference between the mutual information value calculated in thecurrent execution of the iterative closest point registration and mutualinformation registration and a mutual information value calculated inthe previous execution is less than or equal to a predeterminedthreshold. The predetermined threshold may be, for example, 0.001. Inaddition, in the embodiment, the predetermined number of times forrepetition may be in the range of 400 to 800, for example, 450, 500,550, 600, 650, 700, or 750.

If the result of the determination is “No,” the rigid registrationmethod 200 may return to step 230 through stochastic gradient descent.If the result of the determination is “Yes,” the rigid registrationmethod 200 may include outputting the second transformation parametersto perform step 270.

Now referring back to FIG. 1, the image preprocessing step 140 of theimage registration method 100 may optionally (thus shown in dashedboxes) include performing operations of resampling 142 and/ornormalization 144 on the reference image and the floating image beforethe rigid registration 146 based on iterative closest point registrationand mutual information registration is performed thereon.

The resampling 142 enables the reference image and the floating image tohave the same pixel pitch. In an embodiment, the pixel pitch of thereference image and the floating image may be resampled to 1*1*1 throughthe resampling 142. In an embodiment, the resampling 142 may use, forexample, nearest neighbor interpolation, partial volume (PV)interpolation, linear interpolation, B-spline interpolation, bilinearinterpolation, and/or bicubic interpolation.

The normalization 144 can eliminate negative effects of singular pixels.After the normalization, pixel values of the image may fall into astandard normal distribution with a mean of 0 and a variance of 1. In anembodiment, the normalization 144 of both the reference image and thefloating image may be performed based on the following equation (1):

$\begin{matrix}{{x_{norm} = \frac{x - x_{\min}}{x_{\max} - x_{\min}}},} & (1)\end{matrix}$

where x_(norm) is a normalized pixel value, x is an original pixelvalue, and x_(min) and x_(max) are respectively a minimum pixel valueand a maximum pixel value in the original image.

As shown in FIG. 1, after the rigid registration 146, the imagepreprocessing step 140 may optionally further include performingadaptive cropping 148 on the reference image and the rigidly registeredfloating image. The adaptive cropping 148 can reduce interference fromedge background information of the images, thereby improving theprecision of image registration. Further, since the rigid registrationcan align the overall contours of the reference image and the floatingimage, edge portions common to the reference image and the floatingimage can be removed by performing adaptive cropping after the rigidregistration. If cropping is performed before the rigid registration,non-edge portions of the images may be cropped out.

In an embodiment, for images of an object such as an anatomicalstructure (for example, the lung, chest, or abdomen), the adaptivecropping 148 may position a contour of a target object in the referenceimage and the rigidly registered floating image through a semi-automaticor automatic segmentation method. The semi-automatic or automaticsegmentation method may be a deep learning-based segmentation method.Then, an opening operation may be performed on the contour of the targetobject to remove isolated points. After the opening operation, a minimumbounding box of the contour of the target object may be calculated. Inan example of three-dimensional images, the minimum bounding box may bea three-dimensional bounding box, for example, a minimum cube boundingbox.

The minimum bounding box may then be automatically expanded to a desiredsize, so as to obtain a reference image and a floating image that arecropped based on the desired size. The desired size enables the croppedreference image and floating image to be suitable for input to anon-rigid registration model. This is because the non-rigid registrationmodel has size requirements for an input image and the size of theminimum bounding box may not meet such requirements, and thus theminimum bounding box generally cannot be directly input to the non-rigidregistration model. For example, in an exemplary non-rigid registrationmodel where a U-Net contains four convolutional downsampling layers, thesize of the image may be reduced to half of the original size each timeconvolution is performed per layer. As a result, the cropped referenceimage and floating image cannot be input to the U-Net if the size ofeach dimension of the minimum bounding box is not expanded to a multipleof 2⁴ (namely, 16). In this example, each dimension of the minimumbounding box may preferably be expanded to a first reached multiple of16. For example, an exemplary minimum bounding box is 278 pixels*273pixels*236 pixels, which may have a desired size of 288 pixels*288pixels*240 pixels. In other embodiments, for example, when the U-Netcontains two convolutional layers, the size of each dimension of theminimum bounding box may be expanded to a multiple of 2² (namely, 4); asanother example, when the U-Net contains eight convolutional layers, thesize of each dimension of the minimum bounding box may be expanded to amultiple of 2⁸ (namely, 256).

After the image preprocessing 140, the image registration method 100 mayperform non-rigid registration on the preprocessed reference image andfloating image in step 160. Rigid registration of images can achievecoordinate alignment, while non-rigid registration can solve deformationproblems. In some embodiments, the non-rigid registration 160 uses anunsupervised model of a deep learning-based U-Net and a spatialtransformation network (STN). Specifically, the non-rigid registration160 may include inputting the preprocessed reference image and thepreprocessed floating image to the U-Net to obtain spatialtransformation parameters, and then inputting the obtained spatialtransformation parameters to the spatial transformation network (STN),and performing spatial transformation and an interpolation operation onthe preprocessed floating image, so as to obtain a registration resultimage. The spatial transformation parameters may also be referred to asa deformation field. The spatial transformation in the process of rigidregistration 160 can realize fine registration between the referenceimage and the floating image. In an embodiment, the spatialtransformation may include a translation operation on the preprocessedfloating image. The translation operation may include translationoperations of various pixels of the floating image in the x-, y-, and/orz-axis directions. In an embodiment, the interpolation operationperformed on the preprocessed floating image may be, for example,bilinear interpolation.

In the embodiment of the present disclosure, the model used in thenon-rigid registration process may be trained before the non-rigidregistration 160 is performed. An exemplary method 300 for training anon-rigid registration model for images according to an embodiment ofthe present disclosure is described in more detail below with referenceto FIG. 3.

The model training method 300 starts at step 310. In step 310, apreprocessed reference image and a preprocessed floating image may beinput to a U-Net to obtain spatial transformation parameters. Then, instep 330, the obtained spatial transformation parameters may be input toa spatial transformation network (STN), and spatial transformation andan interpolation operation are performed on the preprocessed floatingimage, so as to obtain a registration result image. In an embodiment,the spatial transformation may include a translation operation on thepreprocessed floating image. The translation operation may includetranslation operations of various pixels of the floating image in thex-, y-, and/or z-axis directions. In an embodiment, the interpolationoperation performed on the preprocessed floating image may be, forexample, bilinear interpolation.

Then, in step 350, a loss function value between the obtainedregistration result image and the reference image may be calculated byusing a loss function. The loss function used herein is based onlinearly dependent changes between images. In the case where the lossfunction includes only a correlation coefficient between the referenceimage and the registration result image, the trained model can hardlyachieve a satisfactory registration result for images with largedeformation. The inventor found that the deficiency can be effectivelycompensated for by a mean squared error between the reference image andthe registration result image. Thus, the loss function of the presentdisclosure includes both the correlation coefficient and the meansquared error between the reference image and the registration resultimage, so that the difference between the output registration result andthe reference image can be better measured during training of thenon-rigid registration model, thereby improving the performance of thenon-rigid registration model. In an embodiment, the loss function may berepresented by the following equation (2):

$\begin{matrix}{{L = {{\alpha\frac{\frac{1}{N - 1}{\sum\limits_{i = 0}^{N - 1}\;{( {F_{i} - F^{t}} )( {T_{i} - T^{t}} )}}}{\sqrt{\frac{1}{N - 1}{\sum\limits_{i = 0}^{N - 1}\;( {F_{i} - F^{t}} )^{2}}}\sqrt{\frac{1}{N - 1}{\sum\limits_{i = 0}^{N - 1}( {T_{i} - T^{t}} )^{2}}}}} + {\beta\frac{1}{N - 1}{\sum\limits_{i = 0}^{N - 1}( {F_{i} - T_{i}} )^{2}}}}},} & (2)\end{matrix}$

where L represents the loss function, a is a coefficient of thecorrelation coefficient between the reference image and the registrationresult image, β is a coefficient of the mean squared error between thereference image and the registration result image, N is the number ofpixels of the reference image and the registration result image, F_(i)and T_(i) are respectively values of the i-th pixels of the referenceimage and the registration result image, and F′ and T′ are respectivelyaverage pixel values of the reference image and the registration resultimage. In the embodiment, values of α and β may be set as needed. Forexample, in an example, α may be set to 0.7, and β may be set to 0.3.

After the loss function value is calculated, the training method 300 mayinclude determining whether the non-rigid registration model convergesor a predetermined number of times of iterative training has beenperformed in step 370. In the present disclosure, the model may beconsidered as “converged” if a difference between the loss functionvalue calculated in the current iteration and a loss function valuecalculated in the previous iteration is less than or equal to apredetermined threshold. If the model has converged, it indicates thatthe model has learned a non-linear mapping relationship between thereference image and the floating image as desired, so as to performnon-rigid registration on the reference image and the floating image.

If the determination result in block 370 is “No,” the training method300 may return to step 310. If the determination result in block 370 is“Yes,” the training may end and model parameters may be stored for usein the non-rigid registration process in the image registration process.In the embodiment, the model parameters may include values of variousconvolution kernels of the convolutional layers in the U-Net.

Referring back again to FIG. 1, in step 180, the image registrationmethod 100 may include outputting a registration result image obtainedthrough rigid registration and non-rigid registration.

The execution of the method of the present disclosure should not belimited to the sequence described above. Rather, some steps in themethod of the present disclosure may be performed in a differentsequence or at the same time, or in some embodiments, certain steps maynot be performed. In addition, any step in the method of the presentdisclosure may be performed with a module, unit, circuit, or any othersuitable means for performing these steps.

FIG. 4A and FIG. 5A show image fusion effects of two sets of test imagesbefore the image registration method described herein is performed. FIG.4B and FIG. 5B show image fusion effects of the two sets of test imagesafter the image registration method described herein is performed. Itcan be seen that the precision of image registration can besignificantly improved by using the image registration method of thepresent disclosure.

In addition, the following table shows a comparison between the imageregistration method of the present disclosure and a conventionalnon-rigid registration method, Demons, in respect of evaluation indexessuch as mutual information, mean squared error (MSE) value, andregistration time. The larger a value of mutual information and/or thesmaller a value of mean squared error, the better the effects ofregistration. In addition, the shorter the registration time, the higherthe registration efficiency.

Mutual Registration Method information MSE time (second) Test data 1Demons 1.0430 0.2217 228 Present 1.0530 0.2157 12.5 Disclosure Test data2 Demons 1.0621 0.1815 223 Present 1.1782 0.0760 11.4 Disclosure

FIG. 6 shows an example of an electronic apparatus 600 for performing animage registration method according to an embodiment of the presentdisclosure. The electronic apparatus 600 includes: one or a plurality ofprocessors 620; and a storage device 610, configured to store one or aplurality of programs, wherein when the one or plurality of programs areexecuted by the one or plurality of processors 620, the one or pluralityof processors 620 are caused to implement the method described herein.The processor is, for example, a digital signal processor (DSP), amicrocontroller, an application-specific integrated circuit (ASIC), or amicroprocessor.

The electronic apparatus 600 shown in FIG. 6 is merely an example, andshould not be regarded as any limits to the function and use scope ofthe embodiment of the present disclosure.

As shown in FIG. 6, the electronic apparatus 600 is implemented in theform of a general-purpose computing apparatus. The components of theelectronic apparatus 600 may include, but not limited to, one or aplurality of processors 620, a storage device 610, and a bus 650connecting different system components (including the storage device 610and the processor 620).

The bus 650 represents one or a plurality of types of bus structures,including a memory bus or a memory controller, a peripheral bus, anaccelerated graphics port, a processor, or a local bus using any busstructure in the plurality of bus structures. For example, thesearchitectures include, but not limited to, an Industrial StandardArchitecture (ISA) bus, a Micro Channel Architecture (MAC) bus, anenhanced ISA bus, a Video Electronics Standards Association (VESA) localbus, and a Peripheral Component Interconnect (PCI) bus.

The electronic apparatus 600 typically includes a plurality of computersystem-readable media. These media may be any available medium that canbe accessed by the electronic apparatus 600, including volatile andnon-volatile media as well as removable and non-removable media.

The storage device 610 may include a computer system-readable medium inthe form of a volatile memory, for example, a random access memory (RAM)611 and/or a cache memory 612. The electronic apparatus 600 may furtherinclude other removable/non-removable, and volatile/non-volatilecomputer system storage media. Only as an example, a storage system 613may be configured to read/write a non-removable, non-volatile magneticmedium (not shown in FIG. 6, often referred to as a “hard disk drive”).Although not shown in FIG. 6, a magnetic disk drive configured toread/write a removable non-volatile magnetic disk (for example, a“floppy disk”) and an optical disk drive configured to read/write aremovable non-volatile optical disk (for example, a CD-ROM, a DVD-ROM,or other optical media) may be provided. In these cases, each drive maybe connected to the bus 650 via one or a plurality of data mediuminterfaces. The storage device 610 may include at least one programproduct which has a group of program modules (for example, at least oneprogram module) configured to perform the functions of the embodimentsof the present disclosure.

A program/utility tool 614 having a group of program modules (at leastone program module) 615 may be stored in, for example, the storagedevice 610. Such a program module 615 includes, but is not limited to,an operating system, one or a plurality of applications, other programmodules, and program data. It is possible for each one or a certaincombination of these examples to include implementations of a networkenvironment. The program module 615 typically performs the functionand/or method in any embodiment described in the present disclosure.

The electronic apparatus 600 may also communicate with one or aplurality of peripheral apparatuses 660 (for example, a keyboard, apointing apparatus, and a display 670), may also communicate with one ora plurality of apparatuses enabling a user to interact with theelectronic apparatus 600, and/or communicate with any apparatus (forexample, a network card or a modem) enabling the electronic apparatus600 to communicate with one or a plurality of other computingapparatuses. Such communication may be carried out via an input/output(I/O) interface 630. Moreover, the electronic apparatus 600 may alsocommunicate with one or a plurality of networks (for example, a localarea network (LAN), a wide area network (WAN) and/or a public network,for example, the Internet) through a network adapter 640. As shown inFIG. 6, the network adapter 640 communicates with other modules of theelectronic apparatus 600 through the bus 650. It should be understoodthat although not shown in the figure, other hardware and/or softwaremodules may be used in conjunction with the electronic apparatus 600,including but not limited to: microcode, a device drive, a redundantprocessing unit, an external magnetic disk drive array, a RAID system, amagnetic tape drive, a data backup storage system, and the like.

The processor 620 performs various functional applications and dataprocessing by running the program stored in the storage device 610.

FIG. 7 schematically shows a block diagram of an exemplary imageregistration apparatus 700 according to an embodiment of the presentdisclosure. The image registration apparatus 700 includes an obtainingdevice 720, a rigid registration device 746, a non-rigid registrationdevice 760, and an output device 780. The obtaining device 720 may beconfigured to obtain a reference image and a floating image to beregistered. The rigid registration device 746 may be configured toperform a rigid registration method for images based on iterativeclosest point registration and mutual information registration describedherein. The non-rigid registration device 760 may be configured toperform non-rigid registration on the preprocessed reference image andthe preprocessed floating image to obtain a registration result image.The output device 780 may be configured to output the obtainedregistration result image.

In an embodiment, the image registration apparatus 700 may optionally(thus shown in dashed boxes) include a resampling device 742 and anormalization device 744. The resampling device 742 may be configured toresample the reference image and the floating image so that thereference image and the floating image have the same pixel pitch. Thenormalization device 744 may be configured to normalize the referenceimage and the floating image. In an embodiment, the image registrationapparatus 700 may further optionally include an adaptive cropping device748. The adaptive cropping device 748 may be configured to performadaptive cropping on the reference image and the rigidly registeredfloating image.

Referring to FIG. 7, the rigid registration device 746 may furtherinclude an iterative closest point registration module, a mutualinformation registration module, an image registration module, andoptionally an extraction module and an iteration module. The iterativeclosest point registration module may be configured to perform coarseregistration about contour point data sets on a reference image and afloating image to be registered by using iterative closest pointregistration, so as to obtain first transformation parameters. Themutual information registration module may be configured to optimizeregistration between the reference image and the floating image by usingmutual information registration based on the obtained firsttransformation parameters, so as to obtain second transformationparameters. The image registration module may be configured to registerthe reference image and the floating image based on the obtained secondtransformation parameters. The extraction module may be configured toextract respective contour point data sets of the reference image andthe floating image to be registered. The iteration module may beconfigured to repeatedly perform operations of the iterative closestpoint registration module and the mutual information registration modulethrough stochastic gradient descent a predetermined number of times oruntil a calculated mutual information value converges.

In addition, the non-rigid registration device 760 may further include aU-Net module, an STN module, a loss function value calculation module,and an iteration module. The U-Net module may be configured to input apreprocessed reference image and a preprocessed floating image to aU-Net to obtain spatial transformation parameters. The STN module may beconfigured to input the obtained spatial transformation parameters to aspatial transformation network (STN), and perform spatial transformationand an interpolation operation on the preprocessed floating image, so asto obtain a registration result image. The loss function valuecalculation module may be configured to calculate a loss function valuebetween the obtained registration result image and the reference imageby using a loss function. The iteration module may be configured torepeat various operations of non-rigid registration while training anon-rigid registration model, until the non-rigid registration modelconverges or a predetermined number of times of iterative training hasbeen performed.

According to an embodiment of the present disclosure, acomputer-readable medium is further provided, having instructionsthereon, and when executed by a processor, the instructions cause theprocessor to perform the steps of the method of the present disclosure.The computer-readable medium may include, but not limited to, anon-transitory, tangible arrangement of an article manufactured orformed by a machine or apparatus, including a storage medium such as thefollowing: a hard disk; any other type of disk including a floppy disk,an optical disk, a compact disk read-only memory (CD-ROM), a compactdisk rewritable (CD-RW), and a magneto-optical disk; a semiconductordevice such as a read-only memory (ROM), a random access memory (RAM)such as a dynamic random access memory (DRAM) and a static random accessmemory (SRAM), an erasable programmable read-only memory (EPROM), aflash memory, and an electrically erasable programmable read-only memory(EEPROM); a phase change memory (PCM); a magnetic or optical card; orany other type of medium suitable for storing electronic instructions.The computer-readable medium may be installed in a CT apparatus, or maybe installed in a separate control apparatus or computer that remotelycontrols the CT apparatus.

FIG. 8 shows a block diagram of an exemplary system 800 for imageregistration according to an embodiment of the present disclosure.Referring to FIG. 8, the system 800 may include a medical imagingapparatus 801 configured to perform an imaging scan to generate amedical image, a storage apparatus 802 configured to store the medicalimage, and a medical imaging workstation 803 or a medical image cloudplatform analysis system 804 communicatively connected to the storageapparatus 802 and including a processor 805. The processor 805 may beconfigured to perform various methods described herein.

The medical imaging apparatus 801 may be a CT apparatus, an MRIapparatus, a C-arm imaging apparatus, a PET apparatus, a SPECTapparatus, or any other suitable imaging apparatus.

The storage apparatus 802 may be located in the medical imagingapparatus 801, in a server external to the medical imaging apparatus801, in a stand-alone medical image storage system (such as a PACS),and/or in a remote cloud storage system. The medical imaging workstation803 may be disposed locally to the medical imaging apparatus 801, whilethe medical image cloud platform analysis system 804 may be locatedremotely from the medical imaging apparatus 801, for example, at a cloudin communication with the medical imaging apparatus 801. As an example,after a medical institution completes an imaging scan using the medicalimaging apparatus 801, the data obtained from the scan is stored in thestorage apparatus 802; the medical imaging workstation 803 may directlyread the data obtained from the scan and perform image registrationusing the method of the present disclosure through the processorthereof. As another example, the medical image cloud platform analysissystem 804 may read medical images in the storage apparatus 802 throughremote communication so as to provide the “software as a service(SAAS).” The SAAS may exist between hospitals, between a hospital and animaging center, or between a hospital and a third-party online medicalservice provider.

The technology described in the present disclosure may be implemented atleast in part through hardware, software, firmware, or any combinationthereof. For example, aspects of the technology may be implementedthrough one or a plurality of microprocessors, digital signal processors(DSPs), application-specific integrated circuits (ASICs), fieldprogrammable gate arrays (FPGAs), or any other equivalent integrated ordiscrete logic circuits, and any combination of such parts embodied in aprogrammer (such as a doctor or patient programmer, a stimulator, orother apparatus). The term “processor”, “processing circuit”,“controller” or “control module” may generally refer to any of the abovenoted logic circuits (either alone or in combination with other logiccircuits), or any other equivalent circuits (either alone or incombination with other digital or analog circuits).

Example 1 is an image registration method. The method may include:obtaining a reference image and a floating image to be registered;performing image preprocessing on the reference image and the floatingimage, wherein the image preprocessing may include rigid registrationbased on iterative closest point registration and mutual informationregistration; performing non-rigid registration on the preprocessedreference image and the preprocessed floating image to obtain aregistration result image; and outputting the registration result image.

Example 2 includes the method defined in Example 1, wherein the rigidregistration based on iterative closest point registration and mutualinformation registration may further include: performing coarseregistration about contour point data sets on the reference image andthe floating image to be registered by using iterative closest pointregistration, so as to obtain first transformation parameters;optimizing registration between the reference image and the floatingimage by using mutual information registration based on the firsttransformation parameters, so as to obtain second transformationparameters; and registering the reference image and the floating imagebased on the second transformation parameters.

Example 3 includes the method defined in Example 2, wherein a contourpoint data set of each of the reference image and the floating image maybe extracted through a marching cubes algorithm.

Example 4 includes the method defined in Example 2 or 3, wherein themutual information registration may further include: performing spatialtransformation on the floating image by using the first transformationparameters as initial values of the mutual information registration;performing an interpolation operation on the spatially transformedfloating image; and calculating a mutual information value between anobtained interpolation result and the reference image, so as to obtainthe second transformation parameters corresponding to the mutualinformation value.

Example 5 includes the method defined in Example 4, wherein the spatialtransformation may include one or more of a translation operation of thefloating image in an x-axis direction, a translation operation in ay-axis direction, a translation operation in a z-axis direction, arotation operation in an axial plane, a rotation operation in a sagittalplane, and a rotation operation in a coronal plane.

Example 6 includes the method defined in Example 4, wherein theinterpolation operation may include one or more of nearest neighborinterpolation, partial volume interpolation, linear interpolation,B-spline interpolation, bilinear interpolation, and bicubicinterpolation.

Example 7 includes the method defined in Example 4, wherein theoptimization may further include: repeating the iterative closest pointregistration and the mutual information registration through stochasticgradient descent a predetermined number of times or until the mutualinformation value converges.

Example 8 includes the method defined in any example of Examples 2 to 7,wherein each of the first transformation parameters and the secondtransformation parameters may include: a pixel translation amount in anx-axis direction, a pixel translation amount in a y-axis direction, apixel translation amount in a z-axis direction, a central rotation anglein an axial plane, a central rotation angle in a sagittal plane, and acentral rotation angle in a coronal plane; or a pixel translation amountin an x-axis direction, a pixel translation amount in a y-axisdirection, and a central rotation angle in an xy plane.

Example 9 includes the method defined in any example of Examples 1 to 8,wherein the image preprocessing may further include: resampling thereference image and the floating image before the rigid registrationbased on iterative closest point registration and mutual informationregistration is performed, so that the reference image and the floatingimage have the same pixel pitch.

Example 10 includes the method defined in any example of Examples 1 to9, wherein the image preprocessing may further include: normalizing thereference image and the floating image by using the same normalizationequation before the rigid registration based on iterative closest pointregistration and mutual information registration is performed.

Example 11 includes the method defined in any example of Examples 1 to10, wherein the image preprocessing may further include: performingadaptive cropping on the reference image and the floating image afterthe rigid registration based on iterative closest point registration andmutual information registration is performed.

Example 13 includes the method defined in Example 11, wherein theadaptive cropping may further include: positioning a contour of a targetobject in the reference image and the rigidly registered floating imagethrough a semi-automatic or automatic segmentation method; performing anopening operation on the contour to remove isolated points; calculatinga minimum bounding box of the contour subjected to the openingoperation; and automatically expanding the minimum bounding box to adesired size, so as to obtain a reference image and a floating imagethat are cropped based on the desired size, wherein the desired sizeenables the cropped reference image and floating image to be suitablefor input to a model for performing the non-rigid registration.

Example 13 includes the method defined in any example of Examples 1 to12, wherein the image preprocessing may be performed on the referenceimage and the floating image at the same time.

Example 14 includes the method defined in any example of Examples 1 to13, wherein the non-rigid registration may further include: inputtingthe preprocessed reference image and the preprocessed floating image tothe U-Net to obtain spatial transformation parameters; and inputting thespatial transformation parameters to the spatial transformation network,and performing spatial transformation and an interpolation operation onthe preprocessed floating image, so as to obtain the registration resultimage.

Example 15 includes the method defined in Example 14, wherein thespatial transformation may include translation operations of pixels ofthe floating image in an x-, y- and/or z-axis direction.

Example 16 includes the method defined in Example 14 or 15, wherein theinterpolation operation may be bilinear interpolation.

Example 17 is a rigid registration method for images based on iterativeclosest point registration and mutual information registration. Themethod may comprise: performing coarse registration about contour pointdata sets on a reference image and a floating image to be registered byusing iterative closest point registration, so as to obtain firsttransformation parameters; optimizing registration between the referenceimage and the floating image by using mutual information registrationbased on the first transformation parameters, so as to obtain secondtransformation parameters; and registering the reference image and thefloating image based on the second transformation parameters.

Example 18 includes the method defined in Example 17, wherein a contourpoint data set of each of the reference image and the floating image maybe extracted through a marching cubes algorithm.

Example 19 includes the method defined in Example 17 or 18, wherein themutual information registration may further include: performing spatialtransformation on the floating image by using the first transformationparameters as initial values of the mutual information registration;performing an interpolation operation on the spatially transformedfloating image; and calculating a mutual information value between anobtained interpolation result and the reference image, so as to obtainthe second transformation parameters corresponding to the mutualinformation value.

Example 20 includes the method defined in Example 19, wherein thespatial transformation may include one or more of a translationoperation of the floating image in an x-axis direction, a translationoperation in a y-axis direction, a translation operation in a z-axisdirection, a rotation operation in an axial plane, a rotation operationin a sagittal plane, and a rotation operation in a coronal plane.

Example 21 includes the method defined in Example 19, wherein theinterpolation operation may include one or more of nearest neighborinterpolation, partial volume interpolation, linear interpolation,B-spline interpolation, bilinear interpolation, and bicubicinterpolation.

Example 22 includes the method defined in Example 19, wherein theoptimization may further include: repeating the iterative closest pointregistration and the mutual information registration through stochasticgradient descent a predetermined number of times or until the mutualinformation value converges.

Example 23 includes the method defined in any example of Examples 17 to22, wherein each of the first transformation parameters and the secondtransformation parameters may include: a pixel translation amount in anx-axis direction, a pixel translation amount in a y-axis direction, apixel translation amount in a z-axis direction, a central rotation anglein an axial plane, a central rotation angle in a sagittal plane, and acentral rotation angle in a coronal plane; or a pixel translation amountin an x-axis direction, a pixel translation amount in a y-axisdirection, and a central rotation angle in an xy plane.

Example 24 includes the method defined in any example of Examples 17 to23, wherein the reference image and the floating image may be resampledbefore the iterative closest point registration, so that the referenceimage and the floating image have the same pixel pitch.

Example 25 includes the method defined in any example of Examples 17 to24, wherein the reference image and the floating image may be normalizedby using the same normalization equation before the iterative closestpoint registration.

Example 26 is a method for training a non-rigid registration model forimages. The method may include: (a) inputting a preprocessed referenceimage and a preprocessed floating image to a U-Net to obtain spatialtransformation parameters; (b) inputting the spatial transformationparameters to a spatial transformation network, and performing spatialtransformation and an interpolation operation on the preprocessedfloating image, so as to obtain a registration result image; (c)calculating a loss function value between the reference image and theregistration result image by using a loss function, wherein the lossfunction may include both a correlation coefficient and a mean squarederror between the reference image and the registration result image; and(d) repeating steps (a) to (c) until the non-rigid registration modelconverges or a predetermined number of times of iterative training hasbeen performed.

Example 27 includes the method defined in Example 26, wherein thespatial transformation may include translation operations of pixels ofthe floating image in an x-, y- and/or z-axis direction.

Example 28 includes the method defined in Example 26 or 27, wherein theinterpolation operation may be bilinear interpolation.

Example 29 is a computer-readable medium, having instructions thereon,wherein when executed by a processor, the instructions cause theprocessor to perform the steps of any of the methods described above.

Example 30 is an image registration apparatus, wherein the imageregistration apparatus may include a device for implementing the stepsof any of the methods described above.

Example 31 is a system for image registration. The system may include: amedical imaging apparatus, the medical imaging apparatus beingconfigured to perform an imaging scan to generate a medical image; astorage apparatus, the storage apparatus being configured to store themedical image; and a medical imaging workstation or a medical imagecloud platform analysis system, wherein the medical imaging workstationor the medical image cloud platform analysis system may becommunicatively connected to the storage apparatus and include aprocessor, the processor may be configured to perform the steps of anyof the methods described above.

Some exemplary embodiments of the present disclosure have been describedabove. However, it should be understood that various modifications canbe made to the exemplary embodiments described above without departingfrom the spirit and scope of the present disclosure. For example, anappropriate result can be achieved if the described techniques areperformed in a different order and/or if the components of the describedsystem, architecture, apparatus, or circuit are combined in othermanners and/or replaced or supplemented with additional components orequivalents thereof; accordingly, the modified other embodiments alsofall within the protection scope of the claims.

1. An image registration method, comprising: obtaining a reference imageand a floating image to be registered; performing image preprocessing onthe reference image and the floating image, wherein the imagepreprocessing comprises rigid registration based on iterative closestpoint registration and mutual information registration; performingnon-rigid registration on the preprocessed reference image and thepreprocessed floating image to obtain a registration result image; andoutputting the registration result image.
 2. The method according toclaim 1, wherein the rigid registration further comprises: performingcoarse registration about contour point data sets on the reference imageand the floating image by using iterative closest point registration, soas to obtain first transformation parameters; optimizing registrationbetween the reference image and the floating image by using mutualinformation registration based on the first transformation parameters,so as to obtain second transformation parameters; and registering thereference image and the floating image based on the secondtransformation parameters.
 3. The method according to claim 2, wherein acontour point data set of each of the reference image and the floatingimage is extracted through a marching cubes algorithm.
 4. The methodaccording to claim 2, wherein the mutual information registrationfurther comprises: performing spatial transformation on the floatingimage by using the first transformation parameters as initial values ofthe mutual information registration; performing an interpolationoperation on the spatially transformed floating image; and calculating amutual information value between an obtained interpolation result andthe reference image, so as to obtain the second transformationparameters corresponding to the mutual information value.
 5. The methodaccording to claim 4, wherein the optimization further comprises:repeating the iterative closest point registration and the mutualinformation registration through stochastic gradient descent apredetermined number of times or until the mutual information valueconverges.
 6. The method according to claim 2, wherein each of the firsttransformation parameters and the second transformation parameterscomprises: a pixel translation amount in an x-axis direction, a pixeltranslation amount in a y-axis direction, a pixel translation amount ina z-axis direction, a central rotation angle in an axial plane, acentral rotation angle in a sagittal plane, and a central rotation anglein a coronal plane; or a pixel translation amount in an x-axisdirection, a pixel translation amount in a y-axis direction, and acentral rotation angle in an xy plane.
 7. The method according to claim1, wherein the image preprocessing further comprises: resampling thereference image and the floating image before the rigid registration isperformed, so that the reference image and the floating image have thesame pixel pitch.
 8. The method according to claim 1, wherein the imagepreprocessing further comprises: performing adaptive cropping on thereference image and the floating image after the rigid registration isperformed.
 9. The method according to claim 8, wherein the adaptivecropping further comprises: positioning a contour of a target object inthe reference image and the rigidly registered floating image through asemi-automatic or automatic segmentation method; performing an openingoperation on the contour to remove isolated points; calculating aminimum bounding box of the contour subjected to the opening operation;and automatically expanding the minimum bounding box to a desired size,so as to obtain a reference image and a floating image that are croppedbased on the desired size, wherein the desired size enables the croppedreference image and floating image to be suitable for input to a modelfor performing the non-rigid registration.
 10. The method according toclaim 1, wherein the image preprocessing is performed on the referenceimage and the floating image at the same time.
 11. The method accordingto claim 1, wherein the non-rigid registration further comprises:inputting the preprocessed reference image and the preprocessed floatingimage to a U-Net to obtain spatial transformation parameters; andinputting the spatial transformation parameters to a spatialtransformation network, and performing spatial transformation and aninterpolation operation on the preprocessed floating image, so as toobtain the registration result image.
 12. A rigid registration methodfor images based on iterative closest point registration and mutualinformation registration, comprising: performing coarse registrationabout contour point data sets on a reference image and a floating imageto be registered by using iterative closest point registration, so as toobtain first transformation parameters; optimizing registration betweenthe reference image and the floating image by using mutual informationregistration based on the first transformation parameters, so as toobtain second transformation parameters; and registering the referenceimage and the floating image based on the second transformationparameters.
 13. The method according to claim 12, wherein a contourpoint data set of each of the reference image and the floating image isextracted through a marching cubes algorithm.
 14. The method accordingto claim 12, wherein the mutual information registration furthercomprises: performing spatial transformation on the floating image byusing the first transformation parameters as initial values of themutual information registration; performing an interpolation operationon the spatially transformed floating image; and calculating a mutualinformation value between an obtained interpolation result and thereference image, so as to obtain the second transformation parameterscorresponding to the mutual information value.
 15. The method accordingto claim 14, wherein the optimization further comprises: repeating theiterative closest point registration and the mutual informationregistration through stochastic gradient descent a predetermined numberof times or until the mutual information value converges.
 16. The methodaccording to claim 12, wherein the reference image and the floatingimage are resampled before the iterative closest point registration, sothat the reference image and the floating image have the same pixelpitch.
 17. A method for training a non-rigid registration model forimages, comprising: (a) inputting a preprocessed reference image and apreprocessed floating image to a U-Net to obtain spatial transformationparameters; (b) inputting the spatial transformation parameters to aspatial transformation network, and performing spatial transformationand an interpolation operation on the preprocessed floating image, so asto obtain a registration result image; (c) calculating a loss functionvalue between the reference image and the registration result image byusing a loss function, wherein the loss function comprises both acorrelation coefficient and a mean squared error between the referenceimage and the registration result image; and (d) repeating steps (a) to(c) a predetermined number of times for iterative training or until thenon-rigid registration model converges.
 18. The method according toclaim 17, wherein the spatial transformation comprises translationoperations of pixels of the floating image in an x-, y- and/or z-axisdirection.
 19. The method according to claim 17, wherein theinterpolation operation is bilinear interpolation.
 20. (canceled) 21.(canceled)
 22. A system for image registration, comprising: a medicalimaging apparatus, the medical imaging apparatus being configured toperform an imaging scan to generate a medical image; a storageapparatus, the storage apparatus being configured to store the medicalimage; and a medical imaging workstation or a medical image cloudplatform analysis system, the medical imaging workstation or the medicalimage cloud platform analysis system being communicatively connected tothe storage apparatus and comprising a processor, the processor beingconfigured to perform the steps of the method according to any one ofclaims 1 to 17.